|
|
Automatic scheduling, execution and monitoring of computational workflows on distributed systems
Jaroš, Marta ; Corbalan Gonzales, Julita (oponent) ; Martinovič, Jan (oponent) ; Jaroš, Jiří (vedoucí práce)
Automated execution of computational workflows has become a critical issue in achieving high productivity in various research and development fields. Over the last few years, workflows have emerged as a significant abstraction of numerous real-world processes and phenomena, including digital twins, personalized medicine, and simulation-based science in general. Workflow execution can be viewed as an orchestration of multiple tasks with diverse computational requirements and interdependencies, determined by the workflow structure. Due to the complexity of workflows, execution can only be satisfied by remote computing clusters or clouds. As these resources are expensive, workflow scheduling plays a crucial role in the automation process. The primary objective of this thesis is to enable automated and reliable execution of computational workflows. Moldable tasks, defined within these workflows, permit execution across multiple computational resources. This affects both the workflow makespan and computational cost, but not equally due to varying computational efficiency. Consequently, the thesis investigates various approaches to workflow scheduling and execution optimization, focusing on methods based on genetic algorithms. Three optimization approaches-targeting both on-demand and static computational resource allocations-are examined and discussed. The optimization process is supported by a performance database, which is collected on-the-fly and maintains parallel scaling of executed tasks and diverse inputs. The sparsity and incompleteness of the performance database are addressed through different interpolation methods. The proposed approaches demonstrate better utilization of computing resources while allowing prioritization of various optimization criteria, such as workflow makespan and computational cost. The final implementation was experimentally validated using real workflows executed on high-performance computing clusters at the IT4Innovations national supercomputing center. Additionally, this thesis presents the design and development of a comprehensive system for automated workflow scheduling, execution offloading and monitoring, completed with features such as accounting, reporting, and fault tolerance. This system, named k-Dispatch, has been commercialized for the neuroscience market by Brainbox, Ltd.
|
|
|
Acceleration of Axisymetric Ultrasound Simulations
Kukliš, Filip ; Vaverka, Filip (oponent) ; Jaroš, Jiří (vedoucí práce)
The simulation of ultrasound propagation through soft biological tissue has a wide range of practical applications. These include the design of transducers for diagnostic and therapeutic ultrasound, the development of new signal processing and imaging techniques, studying the aberration of ultrasound beams in heterogeneous media, ultrasonic tissue classification, training ultrasonographers to use ultrasound equipment and interpret ultrasound images, model-based medical image registration, and treatment planning and dosimetry for high-intensity focused ultrasound. However, ultrasound simulation presents a computationally difficult problem, as simulation domains are very large compared with the acoustic wavelengths of interest. But if the problem is axisymmetric, the governing equations can also be solved in 2D. This allows running simulations with larger grid size, with less computational resources and in a shorter time. This paper model and implements an acceleration of the Full-wave Nonlinear Ultrasound Simulation in an Axisymmetric Coordinate System implemented in Matlab using Mex Files for FFTW DST and DCT transformations. The axisymmetric simulation was implemented in C++ as an extension to the open source K-WAVE toolbox. The codes were optimized to run using one node of Salomon supercomputer cluster (IT4Innovations, Ostrava, Czechia) with two twelve-core Intel Xeon E5-2680v3 processors. To maximize computational efficiency, several stages of code optimization were performed. First, the FFTs were computed using the real-to-complex FFT from the FFTW library. Compared to the complex-to-complex FFT, this reduced the compute time and memory associated with the FFT by nearly 50%. Also, real-to-real DCTs and DSTs were computed using FFTW library, which ones in Matlab version, had to be invoked from dynamically loaded MEX Files. Second, to save memory bandwidth, all operations were computed in single precision. Third, element-wise operations were parallelized using OpenMP and then optimized using streaming SIMD extensions (SSE). The overall computation of the C++ k-space model is up to 34-times faster and uses less than one-third of the memory than Matlab version. The simulation which would take nearly two days by Matlab implementation can be now computed in one and half hour. This all allows running the simulation on the computational grid with 16384 × 8192 grid points within a reasonable time.
|
|
|
Analysis of Operational Data and Detection od Anomalies during Supercomputer Job Execution
Stehlík, Petr ; Nikl, Vojtěch (oponent) ; Jaroš, Jiří (vedoucí práce)
Using the full potential of an HPC system can be difficult when such systems reach the exascale size. This problem is increased by the lack of monitoring tools tailored specifically for users of these systems. This thesis discusses the analysis and visualization of operational data gathered by Examon framework of a high-performance computing system. By applying various data mining techniques on the data, deep knowledge of data can be acquired. To fully utilize the acquired knowledge a tool with a soft-computing approach called Examon Web was made. This tool is able to detect anomalies and unwanted behaviour of submitted jobs on a monitored HPC system and inform the users about such behaviour via a simple to use web-based interface. It also makes available the operational data of the system in a visual, easy to use, manner using different views on the available data. Examon Web is an extension layer above the Examon framework which provides various fine-grain operational data of an HPC system. The resulting soft-computing tool is capable of classifying a job with 84 % success rate and currently, no similar tools are being developed. The Examon Web is developed using Angular for front-end and Python, accompanied by various libraries, for the back-end with the usage of IoT technologies for live data retrieval.
|
|
|
Akcelerace aplikací na GPU v jazyce Python
Turcel, Matej ; Jaroš, Jiří (oponent) ; Jaroš, Marta (vedoucí práce)
Konvenčne sa v oblasti high performance computing (HPC) používajú prekladané jazyky, ako napríklad C++. Skriptovacie jazyky ako Python sú však pohodlnejšie a vývoj aplikácií je v nich rýchlejší a jednoduchší. Táto práca porovnáva jazyky C++ a Python z hľadiska možnosti akcelerácie výpočtov na grafickej karte. Jej cieľom je ukázať, že skriptovacie jazyky sú taktiež použiteľné na implementáciu HPC aplikácií a poukázať na ich výhody a nevýhody oproti prekladaným jazykom. Za týmto účelom je implementovaných niekoľko programov. Tie pozostávajú z niekoľkých menších testovacích programov a jedného väčšieho programu, riešiaceho výpočtovo náročný problém. Implementácie týchto programov v jazykoch C++ a Python sú porovnané ako z hľadiska výkonu, tak z hľadiska náročnosti implementácie.
|
|
|
Návrh binárních amplitudových hologramů pro optické generování ultrazvuku akcelerovaný pomocí GPU
Knotek, Martin ; Vaverka, Filip (oponent) ; Jaroš, Jiří (vedoucí práce)
V této práci se zabýváme možnostmi urychlení vědeckých výpočtů s použitím grafických výpočetních jednotek. Termínem vědecký výpočet v tomto kontextu rozumíme specifický algoritmus, který počítá povrch binárních hologramů, jež se používají při generování ultrazvuku. Zaměříme se na návrh hologramu, zvláště pak na rychlost, se kterou můžeme vypočítat povrch takového hologramu. Za tímto účelem použijeme dvě populární platformy pro paralelní zpracování dat - CUDA a OpenMP. Výsledný povrch hologramu je důležitý, protože ovlivňuje specifické fyzikální vlastnosti hologramu.
|
|
|
Distributed Cluster Management
Bůbela, Vojtěch ; Olšák, Ondřej (oponent) ; Jaroš, Jiří (vedoucí práce)
The main goal of my bachelors thesis is to build and manage a distributed computing cluster. The secondary goal is to ensure that the resource of the cluster are assigned correctly and that a job submitted by the user cannot consume more resources than it was given. I solved this problem by installing and configuring a task scheduler software on multiple compute nodes and one head node. When choosing the task scheduler I considered Slurm and PBS. I compared these two by installing and configuring them on a virtual machine cluster. After consideration i decided to go with the Slurm task scheduler. I installed it on 3 raspberry pi 3B computers using ansible and configured basic functionality. The next step was to configure correct assignment and control of resources and create set of tasks that could demonstrate that I managed to meet the goals of my thesis. The result of my thesis is a distributed computing cluster with a configuration that satisfies the main goal of my thesis. The secondary goal was also met fully.
|
|
|
Analysis of Operational Data and Detection od Anomalies during Supercomputer Job Execution
Stehlík, Petr ; Nikl, Vojtěch (oponent) ; Jaroš, Jiří (vedoucí práce)
Using the full potential of an HPC system can be difficult when such systems reach the exascale size. This problem is increased by the lack of monitoring tools tailored specifically for users of these systems. This thesis discusses the analysis and visualization of operational data gathered by Examon framework of a high-performance computing system. By applying various data mining techniques on the data, deep knowledge of data can be acquired. To fully utilize the acquired knowledge a tool with a soft-computing approach called Examon Web was made. This tool is able to detect anomalies and unwanted behaviour of submitted jobs on a monitored HPC system and inform the users about such behaviour via a simple to use web-based interface. It also makes available the operational data of the system in a visual, easy to use, manner using different views on the available data. Examon Web is an extension layer above the Examon framework which provides various fine-grain operational data of an HPC system. The resulting soft-computing tool is capable of classifying a job with 84 % success rate and currently, no similar tools are being developed. The Examon Web is developed using Angular for front-end and Python, accompanied by various libraries, for the back-end with the usage of IoT technologies for live data retrieval.
|
|
|
Acceleration of Axisymetric Ultrasound Simulations
Kukliš, Filip ; Vaverka, Filip (oponent) ; Jaroš, Jiří (vedoucí práce)
The simulation of ultrasound propagation through soft biological tissue has a wide range of practical applications. These include the design of transducers for diagnostic and therapeutic ultrasound, the development of new signal processing and imaging techniques, studying the aberration of ultrasound beams in heterogeneous media, ultrasonic tissue classification, training ultrasonographers to use ultrasound equipment and interpret ultrasound images, model-based medical image registration, and treatment planning and dosimetry for high-intensity focused ultrasound. However, ultrasound simulation presents a computationally difficult problem, as simulation domains are very large compared with the acoustic wavelengths of interest. But if the problem is axisymmetric, the governing equations can also be solved in 2D. This allows running simulations with larger grid size, with less computational resources and in a shorter time. This paper model and implements an acceleration of the Full-wave Nonlinear Ultrasound Simulation in an Axisymmetric Coordinate System implemented in Matlab using Mex Files for FFTW DST and DCT transformations. The axisymmetric simulation was implemented in C++ as an extension to the open source K-WAVE toolbox. The codes were optimized to run using one node of Salomon supercomputer cluster (IT4Innovations, Ostrava, Czechia) with two twelve-core Intel Xeon E5-2680v3 processors. To maximize computational efficiency, several stages of code optimization were performed. First, the FFTs were computed using the real-to-complex FFT from the FFTW library. Compared to the complex-to-complex FFT, this reduced the compute time and memory associated with the FFT by nearly 50%. Also, real-to-real DCTs and DSTs were computed using FFTW library, which ones in Matlab version, had to be invoked from dynamically loaded MEX Files. Second, to save memory bandwidth, all operations were computed in single precision. Third, element-wise operations were parallelized using OpenMP and then optimized using streaming SIMD extensions (SSE). The overall computation of the C++ k-space model is up to 34-times faster and uses less than one-third of the memory than Matlab version. The simulation which would take nearly two days by Matlab implementation can be now computed in one and half hour. This all allows running the simulation on the computational grid with 16384 × 8192 grid points within a reasonable time.
|
|
|
Akcelerace aplikací na GPU v jazyce Python
Turcel, Matej ; Jaroš, Jiří (oponent) ; Jaroš, Marta (vedoucí práce)
Konvenčne sa v oblasti high performance computing (HPC) používajú prekladané jazyky, ako napríklad C++. Skriptovacie jazyky ako Python sú však pohodlnejšie a vývoj aplikácií je v nich rýchlejší a jednoduchší. Táto práca porovnáva jazyky C++ a Python z hľadiska možnosti akcelerácie výpočtov na grafickej karte. Jej cieľom je ukázať, že skriptovacie jazyky sú taktiež použiteľné na implementáciu HPC aplikácií a poukázať na ich výhody a nevýhody oproti prekladaným jazykom. Za týmto účelom je implementovaných niekoľko programov. Tie pozostávajú z niekoľkých menších testovacích programov a jedného väčšieho programu, riešiaceho výpočtovo náročný problém. Implementácie týchto programov v jazykoch C++ a Python sú porovnané ako z hľadiska výkonu, tak z hľadiska náročnosti implementácie.
|
|
|
A Comparison of Preconditioning Methods for Saddle Point Problems with an Application to Porous Media Flow Problems
Axelsson, Owe ; Blaheta, Radim ; Hasal, Martin
The paper overviews and compares some block preconditioners for the solution of saddle point systems, especially systems arising from the Brinkman model of porous media flow. The considered preconditioners involve different Schur complements as inverse free Schur complement in HSS (Hermitian - Skew Hermitian Splitting preconditioner), Schur complement to the velocity matrix and finally Schur complement to a regularization block in the augmented matrix preconditioner. The inverses appearing in most of the considered Schur complements are approximated by simple sparse approximation techniques as element-by-element and Frobenius norm minimization approaches. A special interest is devoted to problems involving various Darcy, Stokes and Brinkman flow regions, the efficiency of preconditioners in this case is demonstrated by some numerical experiments.
|
host ::
RSS.